In connection with many aspects of man and machine interaction, it is important to track the motions of parts of a human body. For instance, in virtual reality (xe2x80x9cVRxe2x80x9d) applications, the problem of making a fast, accurate, and economical head-tracker that operates throughout a large workspace is crucial. It is also important for other head-mounted display (xe2x80x9cHMDxe2x80x9d) applications. Extensive research has been devoted to the development of optical, magnetic, acoustic and mechanical tracking systems, but head-trackers are still one of the weakest links in existing virtual-environment systems. The fastest and potentially most accurate trackers are mechanical, but these are typically clumsy and range-restrictive. The largest tracking range has been achieved at the University of North Carolina by optoelectronic methods, but this type of system is extremely expensive and difficult to install, calibrate, and maintain. This type of optical tracker is sometimes referred to as an xe2x80x9cinside-outxe2x80x9d tracker, because a camera that is mounted on the user is aimed out toward light sources mounted on the ceiling. Ultrasonic trackers are inexpensive, but must sacrifice speed to achieve reasonable range and are sensitive to acoustical interference, reflections, and obstructions. Magnetic trackers are the most popular because of their convenience of operation (they don""t even require line of sight), but the maximum range is a few feet and distortions caused by metallic objects can be problematic. For reviews of the existing four head-tracker technologies, see: Meyer, K., Applewhite, H. and Biocca, F., xe2x80x9cA survey of position trackers,xe2x80x9d Presence, 1(2):173-200, Spring 1992; Ferrin, F., xe2x80x9cSurvey of helmet tracking technologies,xe2x80x9d SPIE, 1456, Large-Screen-Projection, Avionic, and Helmet-Mounted Displays: 86-94, 1991; and Bhatnagar, D., Position trackers for head mounted display systems: A survey, technical report, University of North Carolina at Chapel Hill, March 1993, all three of which are incorporated herein by reference.
All of the known trackers (magnetic, optical, mechanical and acoustical) require interaction with another component of the apparatus that is located a distance from the body being tracked. With magnetic trackers, a magnetic field generator is provided, spaced from the tracked body. With an optical or acoustical tracker, light or sound sources are provided at known locations. Mechanical trackers are connected to a reference through an arm-like device. Thus, none provide a self-contained apparatus for mounting on the body to be tracked, which apparatus can track the orientation of the body without interaction with radiation or energy from any other apparatus. Such a self contained tracking apparatus is desirable. As used herein, a self-contained tracking apparatus is one that can track the orientation of a body to which it is mounted, without interaction with radiation, energy, signals, or physical connections from any other apparatus.
Another drawback with acoustic and outside-in optical trackers is that they fundamentally only measure position. Orientation is then computed from the positions of three fixed points on the head. (By xe2x80x9corientationxe2x80x9d it is meant herein the rotational alignment relative to an external reference frame.) Therefore, the angular resolution is limited by the uncertainty in the position measurements as well as the distance between the three fixed points on the head. With 100 mm spacing between the fixed points, a positional jitter of xc2x11.0-mm causes an orientational jitter of up to xc2x11.1xc2x0. Additionally, since the position tracker is essentially part of the angular orientation tracker, it is not possible to meet independent specifications for the orientation tracker relative to the specifications for the position tracker.
It is also important to track other body members for other aspects of man and machine interaction. Most machines require a user instruction input device, typically actuated by the user""s hand. The head, feet, torso and other body parts may also provide input instructions. Persons with special needs, such as paralysis of certain limbs, often use head and leg motions to complete tasks more typically conducted by hand motions.
Inertial navigation systems (xe2x80x9cINSxe2x80x9d) using accelerometers and rate gyroscopes have been used for decades for ships, planes, missiles and spacecraft. Typically, such apparatus have been rather large, at least on the order of 8-10 cm in diameter and twice that in length, weighing on the order of 10 kg. An inertial navigation system is a type of self contained tracking apparatus, as that term is used herein. By xe2x80x9cinertial apparatusxe2x80x9d, it is meant an apparatus that measures its own motion relative to an inertial reference frame through the measurement of acceleration.
A basic type of INS is called Strapdown INS, and consists of three orthogonal accelerometers and three orthogonal rate gyros fixed to the object being tracked. The orientation of the object is computed by jointly integrating the outputs of the rate gyros (or angular rate sensors), whose outputs are proportional to angular velocity about each axis. The position can then be computed by double integrating the outputs of the accelerometers using their known orientations. If the actual acceleration is {right arrow over (a)} and the acceleration of gravity is {right arrow over (g)}, then the acceleration measured by the triaxial accelerometers will be {right arrow over (a)}measured={right arrow over (a)}+{right arrow over (g)}. To obtain the position it is necessary to know the direction and magnitude of {right arrow over (g)} relative to the tracked object at all times in order to double integrate {right arrow over (a)}={right arrow over (a)}measuredxe2x88x92{right arrow over (g)}. Detailed information about inertial navigation systems is available in the literature, such as Broxmeyer, C., Inertial Navigation Systems, McGraw-Hill, New York, (1964); Parvin, R., Inertial Navigation, Van Nostrand, Princeton, N.J. (1962); and Britting, K., Inertial Navigation Systems Analysis, Wiley-Interscience, New York (1971), which are incorporated herein by reference.
A difficulty with using gyroscopes for head-orientation tracking is drift. Drift arises from integrating over time, a signal that is noisy, or has a bias. Drift would make the virtual world appear to gradually rotate about the user""s head even when the user is not moving. By measuring the output of an angular rate sensor while it is at rest, it is possible to know its output bias and subtract the bias from all subsequent measurements. However, there is inevitably some random noise produced by the sensor in addition to its bias. In the short term, the angular drift is a random walk with RMS value growing proportional to {square root over (t)}. However, the small bias that remains even in a well-calibrated system leads to a drift error that grows as t, which will eventually exceed the Brownian Motion error that grows as {square root over (t)}.
U.S. Pat. No. 5,181,181, issued on Jan. 19, 1993, to Glynn, for xe2x80x9cComputer Apparatus Input Device for Three-Dimensional Information,xe2x80x9d discloses a computer input mouse, which senses six degrees of motion using three accelerometers for sensing linear translation and three angular rate sensors for sensing angular rotation about three axes. The disclosure does not acknowledge or address the problem of drift.
Complete virtual environment systems also suffer from a problem that is not directly related to the problem of tracking body member motions and orientations. A great deal of graphical rendering is required to present to the user a visual image of the environment being simulated. The view to be presented depends on the user""s orientation and position. The rendering requires significant computation, which is time consuming. Typically, the computation can not begin until the orientation is known. Thus, the speed of information acquisition is extremely important. It would also shorten the overall system latency if a reliable method of predicting the user""s orientation in advance existed.
Thus the several objects of the invention include to track the angular orientation of the head (or other body member) with undiminished performance over an unlimited range or working volume. Another object is to track body member orientation with low latency, thus preserving the illusion of presence and avoiding simulator sickness. Another object is to track body member orientation with low output signal noise, so that it won""t be necessary to reduce jitter through use of delay-ridden filters. Another object is to track body orientation without interference problems. Interference sources to be avoided include acoustical, optical, mechanical and electromagnetic. Another object of the invention is to predict what the orientation of the body member will be a short time in the future, to reduce system delays when the orientation tracker is used with other delay-inducing apparatus, such as a graphics rendering engine in a virtual environment simulator. An additional object is to resolve body member orientation without Limitation due to the quality or absence of a position sensing device. Yet another object is to facilitate a modular head tracking apparatus, using independent orientation and position tracking modules, thus permitting tailoring each to independent specifications. Another object of the invention is to track the orientation of a body using a self-contained sensing apparatus, so that an unlimited number of trackers may be used in the same area simultaneously without performance degradation.
In a preferred embodiment, the invention is a self contained sensor apparatus for generating a signal that corresponds to at least two of the three orientational aspects of yaw, pitch and roll of a human-scale body, relative to an external reference frame. The apparatus comprises: a self contained sensor for generating first sensor signals that correspond to rotational accelerations or rates of the body about certain axes relative to said body; a mechanism for mounting the sensor to the body and, coupled to the sensor, a signal processor for generating orientation signals relative to the external reference frame that correspond to the angular rate or acceleration signals, wherein the first sensor signals are impervious to interference from electromagnetic, acoustic, optical and mechanical sources.
In a preferred embodiment that uses rate sensors, the signal processor also includes an integrator to integrate the rate signal over time. The rate sensors may be vibrating piezoelectric devices, silicon micro-machined devices, magneto-hydrodynamic devices or optical devices, to name several.
Another preferred embodiment of the invention further includes a drift compensator, coupled to the angular rate sensor and the integrator, for compensating for any drift with respect to time in the rotational orientation signal.
According to one preferred embodiment, the drift compensator may include a gravitational tilt sensor, or a magnetic field sensor, or both.
The drift compensator, according to another preferred embodiment, includes a verifier that periodically measures the orientation of the body by a means different from using the rotational rate signal and generates an orientation drift compensation signal based on the verification measurement to reduce the effect of drift.
The verifier may take into account characteristic features of human motion, such as the existence of stillness periods. The drift compensator may be implemented using, in part, a Kalman filter, which may utilize statistical data about human head motion.
The apparatus of the invention may also include an orientation predictor, that predicts the orientation in which the body will be a short time in the future.
According to yet another preferred embodiment, the invention is an apparatus for generating a signal that corresponds to the orientation of a human-scale body, relative to a reference frame. The apparatus comprises a self contained first sensor for generating a drift sensitive orientation signal that corresponds to the rotational orientation with respect to at least two degrees of freedom of the body and is impervious to interference from electromagnetic, acoustic, optical and mechanical sources and is subject to drift over time. The apparatus also includes a self contained second sensor for generating a drift compensating orientation signal that corresponds to the rotational orientation with respect to the at least two degrees of freedom of the body and is impervious to interference from electromagnetic, acoustic, optical and mechanical sources and which is relatively impervious to drift over time, and a mounting mechanism for mounting the first sensor and the second sensor to the body. Coupled to said first sensor and said second sensor, a signal corrector means for generating a corrected rotational orientation signal based on said drift sensitive and drift compensating orientation signals.
Another preferred aspect of the invention is a method for generating a signal that corresponds to the orientation of a human-scale body, relative to a reference frame. The method comprises the step of using a first self contained sensor physically coupled to the body, generating a drift sensitive orientation signal that corresponds to the rotational orientation with respect to at least two degrees of freedom of the body and that is impervious to interference from electromagnetic, acoustic, optical and mechanical sources and is subject to drift over time. The method also includes the steps of: using a second self contained sensor physically coupled to the body, generating a drift compensating orientation signal that corresponds to the rotational orientation with respect to said at least two degrees of freedom of the body and that is impervious to interference from electromagnetic, acoustic, optical and mechanical sources and which is relatively impervious to drift over time; and generating a corrected rotational orientation signal based on the drift sensitive and drift compensating orientation signals.
According to another preferred embodiment, the method also includes, when generating the corrected rotational orientation signal, the step of taking into account characteristic aspects of human motion, such as the occurrence of periods of stillness, or statistics about head motions in particular applications.
Still another preferred embodiment of the invention is an apparatus for simulating a virtual environment that is displayed to a user. The apparatus comprises a self contained orientation sensor for generating an orientation signal, such as described above, and is impervious to interference from electromagnetic, acoustic, optical and mechanical sources, a position sensor, a mechanism for mounting the position and orientation sensors to the body member, a virtual environment generating means a means for displaying virtual environment signals to the user and means for coupling the position sensor and the orientation sensor to the virtual environment means.